feat: add entity coverage as the new Nemo Anonymizer feature - #195
feat: add entity coverage as the new Nemo Anonymizer feature#195memadi-nv wants to merge 35 commits into
Conversation
memadi-nv
changed the title
f63da45 to
6186d0c
Compare
Signed-off-by: memadi <memadi@nvidia.com>
Signed-off-by: memadi <memadi@nvidia.com>
Signed-off-by: memadi <memadi@nvidia.com>
Signed-off-by: memadi <memadi@nvidia.com>
Signed-off-by: memadi <memadi@nvidia.com>
Signed-off-by: memadi <memadi@nvidia.com>
Signed-off-by: memadi <memadi@nvidia.com>
Signed-off-by: memadi <memadi@nvidia.com>
Signed-off-by: memadi <memadi@nvidia.com>
Signed-off-by: memadi <memadi@nvidia.com>
Signed-off-by: memadi <memadi@nvidia.com>
1520569 to
a532a6e
Compare
Separate exhaustive candidate extraction from deterministic coverage filtering and add regression coverage for structured responses and prompt behavior. Signed-off-by: memadi <memadi@nvidia.com>
Signed-off-by: memadi <memadi@nvidia.com>
Signed-off-by: memadi <memadi@nvidia.com>
Signed-off-by: memadi <memadi@nvidia.com>
Signed-off-by: memadi <memadi@nvidia.com>
Signed-off-by: memadi <memadi@nvidia.com>
Signed-off-by: memadi <memadi@nvidia.com>
a532a6e to
f999832
Compare
The refactor that unified the replace/rewrite evaluate() code paths dropped check_rewrite_judge=True from validate_model_alias_references, causing a misconfigured rewrite_judge alias to go undetected until LLM calls were made. Pass check_rewrite_judge=is_rewrite to restore pre-refactor behaviour. Signed-off-by: memadi <memadi@nvidia.com>
| provider: nvidia | ||
| inference_parameters: | ||
| max_parallel_requests: 16 | ||
| max_tokens: 16384 |
There was a problem hiding this comment.
Why did we select this value for max_tokens?
There was a problem hiding this comment.
Just updated the model name to match the one in build not inference. Regarding the max_token, this is the set value for nemotron-super in build. Should I bump this down for any limitation that I was not aware of?
| # Deliberately does NOT include generic content descriptors (festival, summit, club, | ||
| # conference, …): those turn a named event/org into a quasi-identifier, so ignoring | ||
| # them would suppress real leaks (e.g. "Davos Summit" collapsing into "Davos"). | ||
| _COVERAGE_IGNORE_TOKENS = frozenset( |
There was a problem hiding this comment.
I understand the intent is to remove only grammatical noise, but can we safely assume these tokens never carry meaning within an entity value? For example, they are significant in names such as The Hague, Bank of America, and AT&T—and AT&T would reduce to the single token t. Could this normalization be limited to specific patterns, such as an optional leading article, instead of removing these tokens from arbitrary positions?
There was a problem hiding this comment.
Good catch. Just restricted the normalization to leading articles only (a, an, the) and dropped all prepositions from the set — so "Bank of America" and "AT&T" are no longer at risk of being incorrectly normalized.
The worst case with just leading articles is minor: a value like "The Home Depo" loses its leading "The", but that's a much smaller tradeoff than suppressing real leaks IMO.
| # final entity (adjacency + order required, not merely a shared set of tokens). | ||
| leaked_core = _core_token_sequence(leaked_tokens) | ||
| if leaked_core and any( | ||
| _is_contiguous_sublist(leaked_core, _core_token_sequence(final_tokens)) for final_tokens in final_token_lists |
There was a problem hiding this comment.
Could value-only subspan matching suppress a genuine leak when the same token appears in a different entity or occurrence? For example, a missed surname "Green" would be considered covered by a detected street value "Bowling Green Road". Should matching consider compatible labels or source spans before removing the candidate?
There was a problem hiding this comment.
This is a known limitation documented in the function's docstring. Label-aware matching would be the natural fix, but judge labels are free-form and don't always share the same vocabulary as Anonymizer's detection labels — so comparing them directly would introduce false negatives of a different kind (mismatched labels on the same entity value).
The tradeoff is accepted because the failure mode is bounded: it only affects single-token leaks where that token coincidentally appears inside a longer detected entity of a different type which seems like a very rare case. Also, I'm hoping on the anonymizer detector to catch the single token entity in the first place considering the context- with entity augmentor.
That said, happy to revisit if you feel the risk of suppressing real leaks outweighs the label incompatibility tradeoff.
Signed-off-by: memadi <memadi@nvidia.com>
Co-authored-by: lipikaramaswamy <31832945+lipikaramaswamy@users.noreply.github.com>
Signed-off-by: memadi <memadi@nvidia.com>
Signed-off-by: memadi <memadi@nvidia.com>
Signed-off-by: memadi <memadi@nvidia.com>
Signed-off-by: memadi <memadi@nvidia.com>
Signed-off-by: memadi <memadi@nvidia.com>
Signed-off-by: memadi <memadi@nvidia.com>
Signed-off-by: memadi <memadi@nvidia.com>
|
|
||
|
|
||
| class EntityCoverageSchema(BaseModel): | ||
| leaked_entities: list[LeakedEntity] = Field( |
There was a problem hiding this comment.
Should leaked_entities be required rather than using default_factory=list? As written, {} is valid according to the generated schema, and _parse_leaked_entities({}) returns an empty list. So an incomplete structured response can produce perfect coverage instead of an unavailable score. Removing the default and adding a regression test for {} would distinguish “the judge found no leaks” from “the judge omitted its result.”
|
|
||
| ### Entity Coverage | ||
|
|
||
| > "Which sensitive values from the original text survived into the anonymized output?" |
There was a problem hiding this comment.
The documented meaning still does not match the implementation. This workflow identifies sensitive values in the original text that are absent from final_entities; it never receives the anonymized output. Therefore, it measures detection coverage rather than whether sensitive values survived anonymization. Could we either describe and name it consistently as a detection-recall metric?
|
|
||
| - **`entity_labels`** — the detection taxonomy in scope; the judge only reports values whose type falls within it. | ||
| - **`data_summary`** — used purely to interpret literal values and their semantic types, never to invent entities absent from the text. | ||
| - **`strict_entity_protection`** — (rewrite only) when enabled, the judge also reports inferable/indirect sensitive values, not just literal identifiers. |
There was a problem hiding this comment.
Strict mode is now explicitly limited to literal quasi-identifiers in the prompt, but the documentation still says that it reports inferable or indirect sensitive values. Just thinking, is that what we want?
| "Hand-built or legacy results need their `replace_method` or `rewrite_config` " | ||
| "attribute set before calling evaluate()." | ||
| ) | ||
| try: |
There was a problem hiding this comment.
Should evaluation model validation account for compute_detection_validity? Detection validity is disabled by default, but check_evaluate=True currently validates its model alias unconditionally. A configuration containing only the models needed for the enabled judges can therefore fail before evaluation begins.
2 participants
Summary
Changes
New: Entity Coverage Judge (
engine/evaluation/entity_coverage_judge.py)LLM extraction phase — the judge independently scans the original (pre-anonymization) text and identifies all in-scope sensitive values that appear in the anonymized output. It is scoped by
entity_labels(the detection taxonomy the user configured — e.g.["first_name", "email"]; whenNone, all PII types are in scope),data_summary(semantic context), andstrict_entity_protection(rewrite-only: also flags inferable/indirect values, not just literal identifiers).Deterministic postprocessing phase — before any result is surfaced, a pure-Python filter removes false positives from the LLM output:
_filter_nonliteral_entities): drops candidates whosevaluefield does not appear verbatim in the original text — guards against the LLM hallucinating entities._deduplicate_judge_entities): collapses duplicate candidates._filter_covered_leaked_entities): removes candidates already accounted for by Anonymizer's final entities via three matching modes:"AliceSmith"covered by["Alice", "Smith"]).Output columns —
entity_coverage(float:n_final / (n_final + n_leaked),1.0= nothing leaked,None= judge unavailable) andleaked_entities(list of{value, label, reasoning}dicts).Model role —
entity_coverage_judge, defaults tonemotron-superinevaluate.yaml. Research log on model selection here.EvaluateConfig: detection validity is now opt-incompute_detection_validity: bool = FalsetoEvaluateConfig.EvaluateConfig(compute_detection_validity=True). This is an internal model/threshold diagnostic, not a customer-facing metric.EvaluateConfig.Interface (
anonymizer.py,results.py,display.py)Anonymizer.evaluate()now acceptsconfig: EvaluateConfig | Noneand routescompute_detection_validityto the underlying judge set.entity_coverageandleaked_entitiesare included inAnonymizerResultcolumns and surfaced indisplay_record().Docs (
docs/concepts/evaluation.md,docs/concepts/models.md)1.0, judge failure →None), and the two-phase (LLM + postprocessing) design.entity_coverage_judge.Skill (
skills/anonymizer/SKILL.md)EvaluateConfigknob.Tests
tests/engine/test_entity_coverage_judge.py— unit tests for the postprocessing helpers (exact/subspan/composite matching, non-literal filter, deduplication) and the full workflow.tests/interface/test_anonymizer_interface.py— integration coverage for the new evaluate path.tests/interface/test_anonymizer_logging.py— logging assertions for the new coverage columns.Type of Change
Testing
make testpasses locallymake checkpasses locally (format + lint + typecheck + lock-check)Documentation
make docs-buildpasses locallyRelated Issues
Closes #193